test(map): add benchmarks for v1 and q10 map parsing and rendering by allenporter · Pull Request #970 · Python-roborock/python-roborock

allenporter · 2026-09-28T03:04:53Z

Description

Adds a comprehensive benchmark and profiling suite for Roborock V1 and Q10 map parsing in tests/map/test_benchmarks.py.

Capabilities

Pytest & CodSpeed Integration: Runs as fast unit assertions during standard pytest, and automatically collects high-precision wall-time/callgraph benchmarks when run with pytest --codspeed.
Standalone CLI Runner & Profiler: Can be run directly via python -m tests.map.test_benchmarks or with --profile to print latency percentiles (min, median, mean, p95), ops/sec, and cProfile hotspot breakdowns.
Pixel & Integrity Validation: Strictly verifies image dimensions, format (PNG), color mode (RGBA), and non-empty bounding boxes on every rendered benchmark output outside the timed loop to guarantee benchmarked changes preserve visual output correctness.

Workloads & Permutations Benchmarked

Scale Variations:
- V1 S6 at scale 1 (411x336 px) vs scale 2 (822x672 px) vs scale 4 (1644x1344 px).
- Q10 200x200 grid at scale 1 (200x200 px) vs scale 4 (800x800 px).
Drawables Permutations:
- V1 S6 with all drawables disabled (drawables=[]) vs default drawables vs all available drawables (list(Drawable)).
Payload Workloads:
- Real Roborock V1 S5 map with 2 rooms (s5_fw2008_with_segments.bin).
- Real Roborock V1 S6 map with 6 rooms and active segment (s6_fw2652_with_active_segment_and_no_mop_zone.bin).
- Q10 wire packet decompression & metadata unpack (b01_q10_map.bin).
- Q10 small map packet decoding and PNG rendering (b01_q10_map.bin).
- Q10 full-scale 200x200 multi-room composite layer render.

Baseline Benchmark Results

Benchmark Target	Image Size	Min (ms)	Median (ms)	Mean (ms)	P95 (ms)	Throughput
V1 S5 Map (Scale 4, Default)	1024x964	53.7 ms	54.4 ms	54.9 ms	59.4 ms	18.2 ops/s
V1 S6 Map (Scale 4, Default)	1644x1344	111.5 ms	113.5 ms	114.1 ms	117.9 ms	8.8 ops/s
V1 S6 Map (Scale 2)	822x672	53.4 ms	54.9 ms	55.6 ms	61.4 ms	18.0 ops/s
V1 S6 Map (Scale 1)	411x336	40.8 ms	42.1 ms	42.1 ms	43.8 ms	23.7 ops/s
V1 S6 Map (No Drawables)	1644x1344	91.4 ms	94.3 ms	96.2 ms	116.0 ms	10.4 ops/s
V1 S6 Map (All Drawables)	1644x1344	115.6 ms	117.2 ms	117.8 ms	121.6 ms	8.5 ops/s
Q10 Wire Packet (Unpack only)	—	0.008 ms	0.009 ms	0.009 ms	0.012 ms	109,917 ops/s
Q10 Small Map (Scale 1)	32x24	0.195 ms	0.197 ms	0.202 ms	0.263 ms	4,947 ops/s
Q10 200x200 Map (Scale 1)	200x200	21.8 ms	22.2 ms	22.3 ms	23.6 ms	44.7 ops/s
Q10 200x200 Map (Scale 4)	800x800	36.7 ms	37.5 ms	38.1 ms	42.8 ms	26.3 ops/s

Copilot

Copilot review overview

🟡 Changes recommended

The documented CodSpeed workflow lacks its dependency, and invalid iteration counts can crash the CLI.

Review effort: Lite
Findings: 2

Open (2)

Documented CodSpeed command lacks the required pytest plugin · New
Reject non-positive iteration counts before sampling · New

What changed in this PR

Adds benchmark and profiling coverage for V1 and Q10 map parsing and rendering.

Changes:

Adds five pytest/CodSpeed benchmark workloads.
Adds synthetic full-scale Q10 rendering data.
Adds standalone timing and cProfile reporting.

File	Summary	Findings
`tests/map/test_benchmarks.py`	Benchmark tests and standalone profiling runner	Add the CodSpeed development dependency and validate positive `--iterations` values.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Lash-L · 2026-09-28T03:20:02Z

I like where this is heading :)

Trying to think of some edge cases that are worth encoding here:

Different scaling values?
One with all drawables on
one with no drawables on

Should we be comparing the pixels of the images as well?

allenporter · 2026-09-28T03:49:30Z

Thanks for the feedback @Lash-L! I've updated the benchmark suite to incorporate all of those permutations:

Different Scaling Values:
- V1 S6:
  - Scale 1 (411x336 px): 42.1 ms (23.7 ops/s)
  - Scale 2 (822x672 px): 54.9 ms (18.0 ops/s)
  - Scale 4 (1644x1344 px, default): 113.5 ms (8.8 ops/s)
- Q10 (200x200 grid):
  - Scale 1 (200x200 px): 22.2 ms (44.7 ops/s)
  - Scale 4 (800x800 px): 37.5 ms (26.3 ops/s)
Drawables Permutations (V1 S6):
- No drawables: 94.3 ms (10.4 ops/s)
- Default drawables (charger, path, vacuum position): 113.5 ms (8.8 ops/s)
- All drawables (including walls, no-go, mop zones, obstacles): 117.2 ms (8.5 ops/s)
Pixel & Output Integrity Validation:
- Added _validate_image() which runs outside the timed loop to assert PNG format, RGBA mode, expected pixel dimensions, and non-empty pixel bounding boxes (image.getbbox() is not None) for every benchmark target to guarantee that performance optimizations preserve output fidelity.

allenporter

Fixed merge conflicts, can you approve again? Thank you.

Copilot AI lite review requested due to automatic review settings September 28, 2026 03:04

Copilot started reviewing on behalf of allenporter September 28, 2026 03:05 View session

Copilot AI reviewed Sep 28, 2026

View reviewed changes

Comment thread tests/map/test_benchmarks.py Outdated

Comment thread tests/map/test_benchmarks.py

allenporter marked this pull request as draft September 28, 2026 03:34

allenporter force-pushed the perf-map-benchmarks branch 2 times, most recently from 53674bd to 79bba04 Compare October 4, 2026 22:39

allenporter marked this pull request as ready for review October 4, 2026 22:49

allenporter requested a review from Lash-L October 4, 2026 22:49

Lash-L previously approved these changes Oct 5, 2026

View reviewed changes

allenporter added 2 commits October 5, 2026 14:15

test(map): add benchmarks for v1 and q10 map parsing and rendering

268ab5b

test(map): add scale and drawables permutations and pixel validation

ee6e680

allenporter dismissed Lash-L’s stale review via ee6e680 October 5, 2026 14:16

allenporter force-pushed the perf-map-benchmarks branch from 79bba04 to ee6e680 Compare October 5, 2026 14:16

allenporter commented Oct 5, 2026

View reviewed changes

Lash-L approved these changes Oct 5, 2026

View reviewed changes

allenporter merged commit d54fc90 into Python-roborock:main Oct 5, 2026
9 checks passed

Navigation Menu

Repository navigation

test(map): add benchmarks for v1 and q10 map parsing and rendering - #970

test(map): add benchmarks for v1 and q10 map parsing and rendering#970
allenporter merged 2 commits into
Python-roborock:mainfrom
allenporter:perf-map-benchmarks

allenporter commented Sep 28, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Lash-L commented Sep 28, 2026 •

edited

Loading

Uh oh!

allenporter commented Sep 28, 2026

Uh oh!

allenporter left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

allenporter commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Capabilities

Workloads & Permutations Benchmarked

Baseline Benchmark Results

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Lash-L commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

allenporter commented Sep 28, 2026

Uh oh!

allenporter left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

allenporter commented Sep 28, 2026 •

edited

Loading

Lash-L commented Sep 28, 2026 •

edited

Loading